A Randomized Hyperparameter Tuning of Adaptive Moment Estimation Optimizer of Binary Tree-Structured LSTM

Adam (Adaptive Moment Estimation) is one of the promising techniques for parameter optimization deep learning. Because an adaptive learning rate method and easier to use than Gradient Descent. In this paper, we propose a novel randomized search with randomizing parameters beta1 beta2. Random noise generated by normal distribution added beta2 every step updating function called. experiment, have implemented binary tree-structured LSTM adam optimizer function. It turned out that in best case, hyperparameter tuning ranging from 0.88 0.92 0.9980 0.9999 3.81 times faster fixed = 0.999 0.9. Our algorithm independent therefore performs well using other algorithms such as NAG, AdaGrad, RMSProp.